[Filebeat] [auditd]: Support EXECVE events with truncated argument list #30382

adriansr · 2022-02-14T17:15:34Z

What does this PR do?

This modifies Filebeat's auditd ingest pipeline to support parsing of EXECVE events with truncated argument lists.

In a normal EXECVE event, the auditd fields argc(=N) and a0 to aN-1 are present. The pipeline would store the arguments in the process.args array, as well as set process.args_count to N and process.executable to process.args[0].

A truncated EXECVE event usually lacks the argc field, and contains only the last few aNN fields.

In that case, this PR will add the arguments into process.args with a leading warning [... N truncated arguments ...] and will not populate process.executable.

Why is it important?

This PR avoids ingesting an arbitrarily large number of fields in the form aNN, aNN_len and aNN[M], to prevent a mapping explosion leading to large indices:

Could not index event to Elasticsearch: "status"=>400,
"error"=>{
"type"=>"illegal_argument_exception",
"reason"=>"Limit of total fields [10000] has been exceeded"}}

This was partially fixed by #29601, but after it was merged, we observed truncated EXECVE records that were still causing issues.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Supersedes auditd: Store program arguments in process.args array #29601

This modifies Filebeat's auditd pipeline to support parsing of EXECVE records with truncated argument lists. When such a log is found, the arguments will be appended to process.args with a leading entry informing about the truncation. This is to prevent a mapping explosion in Filebeat when a lot of these logs are ingested with the previous pipeline version.

mergify · 2022-02-14T17:15:41Z

This pull request does not have a backport label. Could you fix it @adriansr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-02-14T17:17:59Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

elasticmachine · 2022-02-14T19:09:58Z

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-02-17T14:01:37.132+0000
Duration: 89 min 13 sec

Test stats 🧪

Test	Results
Failed	1
Passed	8498
Skipped	911
Total	9410

Test errors

Expand to view the tests failures

`Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestNewModuleRegistry
{"log.level":"info","@timestamp":"2022-02-17T15:17:55.901Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: nginx (error), nginx (access), mysql (slowlog), mysql (error), system (syslog), system (auth), auditd (log)","ecs.version":"1.6.0"}
    modules_test.go:104: 
        	Error Trace:	modules_test.go:104
        	Error:      	Not equal: 
        	            	expected: []string{"error", "access"}
        	            	actual  : []string{"access", "error"}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,4 +1,4 @@
        	            	 ([]string) (len=2) {
        	            	- (string) (len=5) "error",
        	            	- (string) (len=6) "access"
        	            	+ (string) (len=6) "access",
        	            	+ (string) (len=5) "error"
        	            	 }
        	Test:       	TestNewModuleRegistry
--- FAIL: TestNewModuleRegistry (0.02s)

Steps errors

Expand to view the steps failures

`filebeat-goIntegTest - mage goIntegTest`

Took 5 min 10 sec . View more details here
Description: mage goIntegTest

`filebeat-windows-2016-windows-2016 - mage build unitTest`

Took 2 min 25 sec . View more details here
Description: mage build unitTest

`filebeat-windows-2016-windows-2016 - mage build unitTest`

Took 1 min 33 sec . View more details here
Description: mage build unitTest

`filebeat-windows-2016-windows-2016 - mage build unitTest`

Took 1 min 33 sec . View more details here
Description: mage build unitTest

`gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat`

Took 0 min 2 sec . View more details here
Description: @echo off gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beats-ci-temp/Beats/beats/PR-30382-2

`filebeat-windows-10-windows-10 - mage build unitTest`

Took 3 min 12 sec . View more details here
Description: mage build unitTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

adriansr · 2022-02-17T15:57:33Z

Tests failed due to flaky test, fixed here #30453

…st (#30382) This modifies Filebeat's auditd pipeline to support parsing of EXECVE records with truncated argument lists. When such a log is found, the arguments will be appended to process.args with a leading entry informing about the truncation. This is to prevent a mapping explosion in Filebeat when a lot of these logs are ingested with the previous pipeline version. (cherry picked from commit 79229e7) # Conflicts: # filebeat/module/auditd/log/ingest/pipeline.yml

…st (#30382) This modifies Filebeat's auditd pipeline to support parsing of EXECVE records with truncated argument lists. When such a log is found, the arguments will be appended to process.args with a leading entry informing about the truncation. This is to prevent a mapping explosion in Filebeat when a lot of these logs are ingested with the previous pipeline version. (cherry picked from commit 79229e7)

…h truncated argument list (#30457) * [Filebeat] [auditd]: Support EXECVE events with truncated argument list (#30382) This modifies Filebeat's auditd pipeline to support parsing of EXECVE records with truncated argument lists. When such a log is found, the arguments will be appended to process.args with a leading entry informing about the truncation. This is to prevent a mapping explosion in Filebeat when a lot of these logs are ingested with the previous pipeline version. (cherry picked from commit 79229e7) Co-authored-by: Adrian Serrano <[email protected]>

…h truncated argument list (#30456) * [Filebeat] [auditd]: Support EXECVE events with truncated argument list (#30382) This modifies Filebeat's auditd pipeline to support parsing of EXECVE records with truncated argument lists. When such a log is found, the arguments will be appended to process.args with a leading entry informing about the truncation. This is to prevent a mapping explosion in Filebeat when a lot of these logs are ingested with the previous pipeline version. (cherry picked from commit 79229e7) Co-authored-by: Adrian Serrano <[email protected]>

…nd-k8s-env * upstream/main: fix typos and improve sentences (elastic#30432) Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (elastic#30440) {,x-pack/}auditbeat: replace uses of github.com/pkg/errors with stdlib equivalents (elastic#30321) Spelling fix (elastic#30439) packetbeat/beater: make sure Npcap installation runs before interfaces are needed in all cases (elastic#30438) Add BC about Homebrew no longer being available in 8.0 (elastic#30419) Install gawk as a replacement for mawk in Docker containers. (elastic#30452) Clean up python-related system tests (elastic#30415) Fix TestNewModuleRegistry flakiness (elastic#30453) [Filebeat] [auditd]: Support EXECVE events with truncated argument list (elastic#30382) Set `log.offset` to the start of the reported line in filestream (elastic#30445) clarify SelectedPackageTypes meaning and improve its usage (elastic#30142) [elasticsearch module] serialize shards properties (elastic#30408) Add docs about hints and templates autodiscovery priority (elastic#30343)

…ckaging-docker * upstream/main: (26 commits) Update docker/distribution to 2.8.0 (elastic#30462) Add `parsers` examples to `filestream` reference configuration (elastic#30529) extend documentation about setting orchestrator.cluster fields (elastic#30518) Forward-port 8.0.1 changelog to main (elastic#30522) Switch skip to use `CI` (elastic#30512) packetbeat/beater: don't attempt to install npcap when already installed (elastic#30509) Fix Docker module: rename fields on dashboards (elastic#30500) fix typos and improve sentences (elastic#30432) Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (elastic#30440) {,x-pack/}auditbeat: replace uses of github.com/pkg/errors with stdlib equivalents (elastic#30321) Spelling fix (elastic#30439) packetbeat/beater: make sure Npcap installation runs before interfaces are needed in all cases (elastic#30438) Add BC about Homebrew no longer being available in 8.0 (elastic#30419) Install gawk as a replacement for mawk in Docker containers. (elastic#30452) Clean up python-related system tests (elastic#30415) Fix TestNewModuleRegistry flakiness (elastic#30453) [Filebeat] [auditd]: Support EXECVE events with truncated argument list (elastic#30382) Set `log.offset` to the start of the reported line in filestream (elastic#30445) clarify SelectedPackageTypes meaning and improve its usage (elastic#30142) [elasticsearch module] serialize shards properties (elastic#30408) ...

Prevents the indices exceeding the 10,000 field limit due to an arbitrarily large number of aNN fields. This is a combination of the following Filebeat module fixes: - elastic/beats#29601 - elastic/beats#30382 Updates version to 2.1.0

…2730) Prevents the indices exceeding the 10,000 field limit due to an arbitrarily large number of aNN fields. This is a combination of the following Filebeat module fixes: - elastic/beats#29601 - elastic/beats#30382 Updates version to 2.1.0

adriansr added 2 commits February 14, 2022 17:57

Changelog entry

3fbb6df

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2022

mergify bot assigned adriansr Feb 14, 2022

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Feb 14, 2022

adriansr added backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify bug review Team:Security-External Integrations labels Feb 14, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2022

mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Feb 14, 2022

adriansr added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label Feb 14, 2022

adriansr requested review from andrewkroh and a team February 14, 2022 19:03

andrewkroh approved these changes Feb 16, 2022

View reviewed changes

Merge branch 'main' into fix_fb_auditd_exec_truncated

e28978c

adriansr merged commit 79229e7 into elastic:main Feb 17, 2022

adriansr deleted the fix_fb_auditd_exec_truncated branch February 17, 2022 17:18

mergify bot mentioned this pull request Feb 17, 2022

[7.17](backport #30382) [Filebeat] [auditd]: Support EXECVE events with truncated argument list #30455

Closed

mergify bot mentioned this pull request Feb 17, 2022

[8.0](backport #30382) [Filebeat] [auditd]: Support EXECVE events with truncated argument list #30456

Merged

mergify bot mentioned this pull request Feb 17, 2022

[8.1](backport #30382) [Filebeat] [auditd]: Support EXECVE events with truncated argument list #30457

Merged

adriansr mentioned this pull request Feb 22, 2022

[Auditd] Populate process.args array with process arguments elastic/integrations#2730

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Filebeat] [auditd]: Support EXECVE events with truncated argument list #30382

[Filebeat] [auditd]: Support EXECVE events with truncated argument list #30382

adriansr commented Feb 14, 2022 •

edited

Loading

mergify bot commented Feb 14, 2022

elasticmachine commented Feb 14, 2022

elasticmachine commented Feb 14, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

`Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset`

`filebeat-goIntegTest - mage goIntegTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat`

`filebeat-windows-10-windows-10 - mage build unitTest`

`Error signal`

Genuine test errors

adriansr commented Feb 17, 2022

[Filebeat] [auditd]: Support EXECVE events with truncated argument list #30382

[Filebeat] [auditd]: Support EXECVE events with truncated argument list #30382

Conversation

adriansr commented Feb 14, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Related issues

mergify bot commented Feb 14, 2022

elasticmachine commented Feb 14, 2022

elasticmachine commented Feb 14, 2022 • edited by jenkins-beats-ci bot Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset

Steps errors

filebeat-goIntegTest - mage goIntegTest

filebeat-windows-2016-windows-2016 - mage build unitTest

filebeat-windows-2016-windows-2016 - mage build unitTest

filebeat-windows-2016-windows-2016 - mage build unitTest

gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat

filebeat-windows-10-windows-10 - mage build unitTest

Error signal

🐛 Flaky test report

Genuine test errors

🤖 GitHub comments

adriansr commented Feb 17, 2022

adriansr commented Feb 14, 2022 •

edited

Loading

elasticmachine commented Feb 14, 2022 •

edited by jenkins-beats-ci bot

Loading

`Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset`

`filebeat-goIntegTest - mage goIntegTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`filebeat-windows-2016-windows-2016 - mage build unitTest`

`gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat`

`filebeat-windows-10-windows-10 - mage build unitTest`

`Error signal`